Session Title: Multiple Linear Regression with PyTorch
Objective : In this session, we will:
Build a multiple linear regression model using PyTorch.
Use sample data generated programmatically.
Explain each line of code in detail.
Understand the real-world relevance of multiple linear regression.
1. What is Multiple Linear Regression? Theory :
Multiple Linear Regression predicts a target variable (\(y\)) based on multiple independent variables (\(x_1, x_2, \ldots, x_n\)).
The relationship is defined as:
where:
\(w_i\): Weights for each feature.
\(b\): Bias (intercept).
\(y\): Target variable. Real-World Example :
Predicting monthly rent (\(y\)) of a house based on:
Size (sq ft) (\(x_1\)).
Number of bedrooms (\(x_2\)).
Distance to city center (\(x_3\)).
2. sample Data Generation We’ll create a dataset with three features (\(x_1, x_2, x_3\)) and a target (\(y\)) using the formula:
import torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt
# Hyper-parameters
input_size = 3 # Three independent variables
output_size = 1 # One target variable
num_epochs = 100
learning_rate = 0.01
# Generate sampledata
np.random.seed(42)
x_train = np.random.rand(100, 3) * 10 # 100 samples, 3 features
weights = np.array([3.0, 2.0, -1.0]) # True weights
bias = 5.0 # True bias
noise = np.random.randn(100, 1) # Add some noise
# Calculate y using the linear relationship
y_train = np.dot(x_train, weights.reshape(-1, 1)) + bias + noise
Code Explanation :
x_train: 100 samples with 3 features each, randomly generated.
Example: A single row might represent a house with:
Size = 6.2 sq ft.
Bedrooms = 3.
Distance = 8.1 km.
weights: The actual relationship between features and the target.
Size contributes positively (\(+3x_1\)).
Bedrooms contribute positively (\(+2x_2\)).
Distance contributes negatively (\(-x_3\)).
bias: A fixed base value for all predictions.
3. Building the Model Model Definition :
# Define the linear regression model
model = nn.Linear(input_size, output_size)
Why a Linear Model?
What Does nn.Linear Do?
input_size = 3 (three features).
output_size = 1 (one predicted value).
4. Define Loss Function and Optimizer Loss Function :
criterion = nn.MSELoss()
Measures the average squared difference between predicted and actual values.
Smaller MSE = Better predictions. Optimizer :
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)
Adjusts weights based on the gradient of the loss function with respect to each weight.
lr=0.01: Controls how big each adjustment step is.
5. Training the Model Training Loop :
for epoch in range(num_epochs):
# Convert numpy arrays to torch tensors
inputs = torch.from_numpy(x_train).float()
targets = torch.from_numpy(y_train).float()
# Forward pass: Compute predictions
outputs = model(inputs)
loss = criterion(outputs, targets)
# Backward pass: Compute gradients
optimizer.zero_grad()
loss.backward()
optimizer.step()
if (epoch + 1) % 10 == 0:
print(f"Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}")
Line-by-Line Explanation :
PyTorch works with torch.Tensor objects, not numpy.
.float(): Ensures the data is in floating-point format.
outputs = model(inputs): Feeds inputs through the model to get predictions.loss = criterion(outputs, targets): Compares predictions (outputs) to actual values (targets).optimizer.zero_grad(): Clears old gradients.
loss.backward(): Calculates gradients for all weights and bias.
optimizer.step(): Adjusts weights and bias to minimize loss.6. Visualizing Results Plot the True vs. Predicted Values
# Evaluate the model
model.eval()
predicted = model(torch.from_numpy(x_train).float()).detach().numpy()
# Visualization
plt.figure(figsize=(10, 6))
plt.scatter(range(len(y_train)), y_train, color='blue', label='True Values')
plt.scatter(range(len(predicted)), predicted, color='red', label='Predicted Values')
plt.title('True vs Predicted Values')
plt.xlabel('Sample Index')
plt.ylabel('Target Value')
plt.legend()
plt.show()
Explanation :
model.eval(): Switches the model to evaluation mode (disables gradient calculations).
detach(): Prevents tracking gradients for predictions (not needed during testing).
Plot :
Blue dots: Actual values.
Red dots: Predicted values.
7. Saving the Model
torch.save(model.state_dict(), 'multiple_linear_regression.ckpt')
Complete Code
import torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt
# Hyper-parameters
input_size = 3
output_size = 1
num_epochs = 100
learning_rate = 0.01
# Generate sampledata
np.random.seed(42)
x_train = np.random.rand(100, 3) * 10
weights = np.array([3.0, 2.0, -1.0])
bias = 5.0
noise = np.random.randn(100, 1)
y_train = np.dot(x_train, weights.reshape(-1, 1)) + bias + noise
# Linear regression model
model = nn.Linear(input_size, output_size)
# Loss and optimizer
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)
# Train the model
for epoch in range(num_epochs):
inputs = torch.from_numpy(x_train).float()
targets = torch.from_numpy(y_train).float()
outputs = model(inputs)
loss = criterion(outputs, targets)
optimizer.zero_grad()
loss.backward()
optimizer.step()
if (epoch + 1) % 10 == 0:
print(f"Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}")
# Visualize results
model.eval()
predicted = model(torch.from_numpy(x_train).float()).detach().numpy()
plt.figure(figsize=(10, 6))
plt.scatter(range(len(y_train)), y_train, color='blue', label='True Values')
plt.scatter(range(len(predicted)), predicted, color='red', label='Predicted Values')
plt.title('True vs Predicted Values')
plt.xlabel('Sample Index')
plt.ylabel('Target Value')
plt.legend()
plt.show()
# Save the model
torch.save(model.state_dict(), 'multiple_linear_regression.ckpt')
Learning Outcomes
Understand multiple linear regression and how to implement it in PyTorch.
Learn to generate sample data for experimentation.
Visualize true vs. predicted values to evaluate model performance.